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DETAILED ACTION 



This action is responsive to application 10/002,998 filed 11/01/01. 



Claims 1-24 have been examined. 



Priority 



Applicant's claim for domestic priority against application number 60/246,052 filed 
11/06/2000 under 35 U.S.C. 1 19(e) is acknowledged. 



The drawings have not been checked to the extent necessary to determine the 
presence of all possible minor errors. Applicant's cooperation is required in correcting 
any errors of which applicant may become aware in the drawings. 
The drawings are objected to because: 

• Fig. 1, step 110 covers both information transfer directions between items 101 
and 111. There should be a different step/item number for each direction as 
suggested from page 7, line 6 to page 8, line 13. 

• Two different item numbers (e.g. 21 1 , 212) were given to 'content relationship' in 
Fig. 2 when page 10, line 5 suggests one of them would read well as 'feature 
relationship'. 



Drawings 
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• Use of parentheses around figure item numbers (e.g. Fig. 4, items 403, 404 and 
406) and in the specification starting on page 6, line 19 is inconsistent. The 
specification and figures would read well without the parentheses. 

• Item 500 of page 12, line 16 is mislabeled or missing in Fig. 5. 

A proposed drawing correction or corrected drawings are required in reply to the Office 
action to avoid abandonment of the application. The objection to the drawings will not 
be held in abeyance. 

Specification 

The specification has not been checked to the extent necessary to determine the 
presence of all possible minor errors. Applicant's cooperation is required in correcting 
any errors of which applicant may become aware in the specification. 
The disclosure is objected to because of the following informalities: 

- The use of the trademarks WORDNET, MPEG, XML and BIM have been noted 
in this application (specification: page 3, line 17 and page 12, lines 14-20, for 
examples). They should be capitalized wherever they appear and be 
accompanied by the generic terminology. Although the use of trademarks is 
permissible in patent applications, the proprietary nature of the marks should be 
respected and every effort made to prevent their use in any manner which might 
adversely affect their validity as trademarks. 

- 'concepts' on page 1 , line 9 would read well as 'semantic concepts'. 

- 'content node' on page 15, line 14 would read well as 'content node 613'. 
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- The title of the invention is not descriptive. A new title is required that is clearly 
indicative of the invention to which the claims are directed. The following title is 
suggested: Network for Describing Multimedia Information. 
Appropriate correction is required. 

Claim Rejections - 35 (JSC § 101 

35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

Claims 1, 6, 12 and 14-16 are rejected under 35 U.S.C. 101 because the claimed 
invention is directed to non-statutory subject matter. The language of the claims (e.g. 
"concepts", "words", "content", "relationships") raise a question as to whether the claims 
are directed merely to an abstract idea that is not tied to a technological art, 
environment or machine which would result in a practical application producing a 
concrete, useful, and tangible result to form the basis of statutory subject matter under 
35 U.S.C. 101 . For example, if claim 1 was amended to recite a computer-implemented 
method, it will be statutory in most cases since use of technology permits the function of 
the descriptive material to be realized. 

Claim Rejections - 35 USC § 102 

To expedite this application, a complete examination appears below, despite the 
presence of the rejection under 35 U.S.C. 101. 
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The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form 
the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1 ) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

Claims 1 and 3-24 are rejected under 35 U.S.C. 102(e) as being anticipated by 
Bergman et al U.S. Patent Number 6,564,263 (Dated May 13, 2003; Filed December 3, 
1999). 

Regarding claim 1: 

Bergman et al teaches, 

- forming a network having nodes that represent semantic concepts (Fig. 9; column 9, 
lines 39-67, "In one multimedia... is illustrated in FIG. 9"; column 10, lines 11-29, 
"Preferably, each connection... the multimedia content") 

- associating one or more words with one or more of the nodes (column 22, lines 40-67, 
"The video component... for the story") 

- associating multimedia content with one or more of the nodes (Figs. 11-15; column 1 1 , 
lines 1-11, "FIG. 12 illustrates an example... multiple fidelities may exist") 

- representing relationships between the nodes as arcs between associated words and 
arcs between associated multimedia content (Figs. 5-6, 15; column 3, lines 17-51, "It is 
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a further... merging of objects, etc."; column 4, lines 20-27, TIG. 5 is a logical. ..the 
present invention"; column 17, lines 49-59, "FIG. 5 shows that... ends after object D") 
Regarding claim 3: 
Bergman et al further teaches, 

- relationships between semantic concepts and between associated content are based 
at least in part on audio and/or visual feature descriptor values (column 8, lines 42-46, 
"Feature Descriptors: These. ..the transformed data") 

Regarding claim 4: 
Bergman et al further teaches, 

- extracting feature descriptors from multimedia content (column 8, lines 45-46, "Note 
that features... the transformed data"; column 9, lines 26-54, "the spatial or... to only 
metadata") 

- computing similarity measures between descriptor values (column 13, lines 25-39, 
"The fundamental description. ..for computing distances") 

Regarding claim 5 
Bergman et al further teaches, 

- the media network knowledge is represented using the ISO MPEG-7 Description 
Definition Language (column 14, lines 45-67, "The multimedia content.. .for a particular"; 
column 15, lines 1-3, "community (say Satellite... set of DTDs"; column 21, lines 65-67, 
"the InfoPyramid is... to client device"; column 22, lines 1-24, "characteristics and 
user... representation for MPEG-7") 
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Regarding claim 6: 

Bergman et al teaches, 

- accepting a query (column 22, lines 5-9, "While content negotiation... another for 
retrieval") 

- matching the query to the words and multimedia content related to the concepts 
encoded in the media network knowledge representation (column 22, lines 9-16, 
"However, the content... of network/computational resources") 

- navigating the relationship arcs of the concepts associated with matching words and 
multimedia content (column 23, lines 22-53, "As we have... limited display capabilities") 

- retrieving related concepts, words, and multimedia content from the matched nodes or 
related nodes (Figs. 9, 11-14, 17-19; column 22, lines 26-39, "This application 
automatically... the full details") 

Regarding claim 7: 

The rejection of claim 6 is incorporated. Therefore, claim 7 is rejected under the same 
rationale as claim 6. 
Regarding claim 8: 

The rejection of claim 6 is incorporated. Therefore, claim 8 is rejected under the same 
rationale as claim 6. 
Regarding claim 9: 

The rejection of claim 6 is incorporated. Therefore, claim 9 is rejected under the same 
rationale as claim 6. 
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Regarding claim 10: 

Bergman et al teaches, 

- displaying one or more concept nodes and associated words and/or multimedia 
content (column 5, lines 44-62, TIG. 2 depicts another... proper client device"; column 7, 
lines 44-62, "each device or... suitable point therebetween") 

- providing means for allowing a user to select related concepts for viewing (Figs. 1-2; 
column 19, lines 26-49, "a method is... device resolution, etc.") 

Regarding claim 11: 
Bergman et al further teaches, 

- providing means for allowing the user to select concept nodes and associated words 
and/or multimedia content for display on the basis of specific types or values of relations 
to a particular concept node or associated word or multimedia content (column 10, lines 
1-44, "With reference to... scene transition diagram"; column 19, lines 64-67, "7. 
Generate annotations... original multimedia content") 

Regarding claim 12: 
Bergman et al teaches, 

- extracting a subset of nodes, relations, and words and/or multimedia content from an 
encoded media network knowledge representation (column 9, lines 39-54, "In one 
multimedia... to only metadata") 
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Regarding claim 13: 

Bergman et al further teaches, 

- consolidating together concept nodes, relations, words, and/or multimedia content 
(column 9, lines 55-67, "One of the... illustrated in FIG. 9"; column 10, lines 1-10, "With 
reference to ... dependency entity description") 

Regarding claim 14: 
Bergman et al teaches, 

- adding, deleting or modifying concepts, relations, or associated words, multimedia 
content, or descriptors in the encoded media network knowledge representation 
(column 12, lines 48-50, "Each description data... that data type") 

Regarding claim 15: 
Bergman et al teaches, 

- searching the encoded media network knowledge representation (column 23, lines 22- 
34, "As we have... to support search") 

- retrieving words, content, and/or descriptors from the media network knowledge 
representation (column 23, lines 34-36, "they also will... search and retrieval") 

- searching the information repository using the retrieved words, content, and/or 
descriptors (column 23, lines 37-43, "Just as in... of the news") 

Regarding claim 16: 
Bergman et al teaches, 

- describing the multimedia information using words or descriptors (Abstract, "A 
framework is... aggregated multimedia objects") 
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- describing user preferences using words multimedia content, and/or descriptors 
(column 3, lines 37-51, "The description scheme... merging of objects, etc."; column 21, 
lines 65-67, "In one application... to client device"; column 22, lines 1-4, "characteristics 
and user preferences... of Internet content") 

- matching the user preferences with the descriptions of the multimedia information (Fig. 
16; column 11, lines 60-67, "In FIG. 16... the lowest fidelity"; column 12, lines 1-11, 
"level and an... multimedia content source") 

- extracting, retrieving, and/or summarizing the matched multimedia items (column 10, 
lines 38-67, "Fidelity transformation (e.g., 904-909... like to graphics") 

Regarding claim 17: 
Bergman et al teaches, 

- means for forming a network having logical nodes that represent semantic concepts 
(Figs. 1, 9; column 9, lines 39-67, "In one multimedia... is illustrated in FIG. 9"; column 

10, lines 1 1-29, "Preferably, each connection... the multimedia content") 

- means for associating one or more words with one or more of the nodes (column 10, 
lines 15-37, "It should be. ..human being), etc."; column 22, lines 40-67, "The video 
component... for the story") 

- means for associating multimedia content with one or more of the nodes (Figs. 11-15; 
column 10, lines 38-50, "Fidelity transformation (e.g... .the present invention"; column 

11, lines 1-11, "FIG. 12 illustrates an example... multiple fidelities may exist") 

- means for representing relationships between the nodes as arcs between associated 
words and arcs between associated multimedia content (Figs. 5-6, 15; column 3, lines 
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17-51, "It is a further... merging of objects, etc."; column 4, lines 20-27, TIG. 5 is a 
logical.. .the present invention"; column 17, lines 49-59, "FIG. 5 shows that. ..ends after 
object D") 

Regarding claim 18: 

Bergman et al further teaches, 

- means for searching the knowledge encoded in the network (column 23, lines 22-34, 
"As we have. ..to support search") 

Regarding claim 19: 
Bergman et al further teaches, 

- means for browsing the knowledge encoded in the network (column 23, lines 30-53, 
"Our contention is... limited display capabilities") 

Regarding claim 20: 
Bergman et al further teaches, 

- means for updating the knowledge encoded in the network (column 12, lines 48-50, 
"Each description data... that data type") 

Regarding claim 21: 
Bergman et al further teaches, 

- means for summarizing the knowledge encoded in the network (column 10, lines 38- 
67, "Fidelity transformation (e.g., 904-909... like to graphics") 
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Regarding claim 22: 

Bergman et al further teaches, 

- means for querying a multimedia information repository associated with the knowledge 
encoded in the network (column 22, lines 5-9, "While content negotiation... another for 
retrieval") 

Regarding claim 23: 

Bergman et al further teaches, 

- means for personalizing the knowledge encoded in the network for a particular user 
(column 3, lines 37-51, 'The description scheme... merging of objects, etc."; column 21, 
lines 65-67, "In one application... to client device"; column 22, lines 1-4, "characteristics 
and user preferences... of Internet content") 

Regarding claim 24: 
Bergman et al teaches, 

- first instructions for forming a network having logical nodes that represent semantic 
concepts (Figs. 1, 9; column 9, lines 39-67, "In one multimedia... is illustrated in FIG. 9"; 
column 10, lines 11-29, "Preferably, each connection... the multimedia content") 

- second instructions for associating one or more words with one or more of the nodes 
(column 10, lines 15-37, "It should be... human being), etc."; column 22, lines 40-67, 
"The video component... for the story") 

- third instructions for associating multimedia content with one or more of the nodes 
(Figs. 11-15; column 10, lines 38-50, "Fidelity transformation (e.g... .the present 
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invention"; column 11, lines 1-11, "FIG. 12 illustrates an example... multiple fidelities 
may exist") 

- fourth instructions for representing relationships between the nodes as arcs between 
associated words and arcs between associated multimedia content (Figs. 5-6, 15; 
column 3, lines 17-51 , "It is a further... merging of objects, etc."; column 4, lines 20-27, 
"FIG. 5 is a logical. ..the present invention"; column 17, lines 49-59, "FIG. 5 shows 
that... ends after object D") 

Claim Rejections - 35 USC § 103 

To expedite this application, a complete examination appears below, despite the 
presence of the rejection under 35 U.S.C. 101. 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the Office presumes that the subject matter of the 
various claims was commonly owned at the time any inventions covered therein were 
made absent any evidence to the contrary. Applicant is advised of the obligation under 
37 CFR 1 .56 to point out the inventor and invention dates of each claim that was not 
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commonly owned at the time a later invention was made in order for the Office to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 1 03(a). 

Claim 2 is rejected under 35 U.S.C. 103(a) as being unpatentable over Bergman 
etal U.S. Patent Number 6,564,263 (Dated May 13, 2003; Filed December 3, 1999) in 
view of M///er "WordNet: A Lexical Database for English" (November 1995). 
Regarding claim 2: 
Bergman et al teaches, 

- forming a network having nodes that represent semantic concepts (Fig. 9; column 9, 
lines 39-67, "In one multimedia... is illustrated in FIG. 9"; column 10, lines 11-29, 
"Preferably, each connection... the multimedia content") 

- associating one or more words with one or more of the nodes (column 22, lines 40-67, 
"The video component... for the story") 

- associating multimedia content with one or more of the nodes (Figs. 11-15; column 1 1 , 
lines 1-11, "FIG. 12 illustrates an example... multiple fidelities may exist") 

- representing relationships between the nodes as arcs between associated words and 
arcs between associated multimedia content (Figs. 5-6, 15; column 3, lines 17-51, "It is 
a further... merging of objects, etc."; column 4, lines 20-27, "FIG. 5 is a logical. ..the 
present invention"; column 17, lines 49-59, "FIG. 5 shows that... ends after object D") 

- creating relations between semantic concepts on the basis of one or more of: word 
forms and word meaning of associated words (column 5, lines 65-67, "a multimedia 
content... digital form in"; column 6, lines 1-14, "terms of either.. .and non-terminal 
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objects"; column 7, lines 2-43, "Multimedia content typically... or semantics pyramid"; 
column 9, lines 26-38, "the spatial or.. .object, event, etc."; column 12, lines 24-57, "the 
multimedia content... data types T1, T2, T3, etc.") 

However, Bergman et al doesn't explicitly teach creating lexical relations between 
semantic concepts on the basis of one or more of: word forms and word meaning of 
associated words while Miller teaches, 

- creating lexical relations between semantic concepts on the basis of one or more of: 
word forms and word meaning of associated words (page 39, paragraphs 1-2, "Because 
meaningful sentences... to be synonymous") 

Motivation - The portions of the claimed method would have been a highly desirable 
feature in this art for 

• Supporting syntactic categories (Miller, page 40, paragraph 2, "WordNet respects 
the... explication in WordNet") 

• Providing a unified means for describing multimedia content as well as spatial 
and temporal characteristics between multiple objects (Bergman et al, column 2, 
lines 58-67, "Despite the latest... of multimedia objects") 

Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made, to combine Bergman et al with Miller to obtain the invention 
specified in claim 2, a method for encoding knowledge. The modification would have 
been obvious because one of ordinary skill in the art would have been motivated to 
support the ability to describe real world relationships between multiple knowledge and 
language components. 
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Conclusion 



The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure: 

- Chang; US 5263126 A; Automatic expert system 

- Borgida et ah US 5806060 A; Interactive data analysis employing a knowledge base 

- Wical] US 61 12201 A; Virtual bookshelf 

- Kobayashi et al\ US 6493692 B1 ; Information search apparatus and method, and 
computer readable memory 

- Bergman et a/; US 6564263 B1 ; Multimedia content description framework 

- Aslandogan et a/; Using semantic contents and WordNet in image retrieval; 
Proceedings of the 20th annual international ACM SIGIR conference on Research and 
development in information retrieval; July 1997; Vol. 31 Is. SI 

- Miller, WordNet: A Lexical Database for English; Communications of the ACM; 
November 1995; Vol. 38, No. 11; pp 39-41 

- Smith et a/; Visually searching the Web for content; Multimedia, IEEE; Vol. 4, Is. 3; 
July-Sept. 1997; pp 12-20 

- Mohan et a/; Adapting multimedia Internet content for universal access; Multimedia, 
IEEE Transactions on; Vol. 1, Is. 1; March 1999; pp 104-114 

- Suzuki et a/; A similarity retrieval of 3D polygonal models using rotation invariant 
shape descriptors; IEEE International Conference on Systems, Man, and Cybernetics; 
Vol. 4; 8-1 1 Oct. 2000; pp 2946-2952 
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- Ohm\ Flexible solutions for low-level visual feature descriptors in MPEG-7; Consumer 
Electronics, International Conference on; 13-15 June 2000; pp 282-283 

- Yang et a/; A semantic classification and composite indexing approach to robust image 
retrieval; Image Processing, International Conference on; Vol. 1; 1999; pp 134-138 

- Church et a/; Word association norms, mutual information, and lexicography; 
Computational Linguistics; March 1990; Vol. 16, Is. 1 

Any inquiry concerning this communication or earlier communications from the 
Office should be directed to Meltin Bell whose telephone number is 703-305-0362. This 
Examiner can normally be reached on Mon - Fri 7:30 am - 4:30 pm. 

If attempts to reach this Examiner by telephone are unsuccessful, his supervisor, 
Anil Khatri, can be reached on 703-305-0282. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is 703-305- 
3900. /? / , . ^ 





